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Final  Report,  Contract  N00014-14-P-1185  (CDRL  0003,  Data  Item 

C001): 

PDS  Preparation  and  Planning 

H.  Van  Dyke  Parunak,  Ph.D. 

Soar  Technology,  Inc. 


1  Executive  Summary 

PRELIM  (Predictive  Relevance  Estimation  from  Linked  Models)  draws  on  semantic  models 
provided  by  some  members  of  the  PDS  team  to  generate  forecasts  and  estimates  of  value  of 
information  (VOI)  for  use  by  other  members  of  the  team. 

The  central  challenge  in  proactive  decision  support  is  to  anticipate  the  decision  and  information 
needs  of  decision-makers,  in  the  light  of  likely  future  developments  in  a  scenario.  In  many  cases, 
this  anticipation  does  not  require  sophisticated  forecasting,  but  can  be  triggered  by  simple 
decision  rules  (e.g,  “If  a  typhoon  is  moving  toward  the  Philippines,  identify  the  state  of 
Humanitarian  Assistance/Disaster  Recovery  resources  within  a  1000  mile  radius”).  Such  an 
approach  corresponds  to  automation  of  elements  of  recognition-primed  decision-making  [4], 
However,  some  cases  may  go  beyond  patterns  that  are  readily  learned  from  past  experience,  and 
require  a  forecasting  mode  of  reasoning  that  in  people  is  called  explanation-based  reasoning  [11]. 
Several  conditions  that  require  proactivity  and  prediction,  including 

•  The  need  to  preposition  assets,  materiel,  or  information  because  of  intrinsic  timelags; 

•  Indications  of  unlikely  but  high  value  future  events  that  cannot  yet  be  perceived  (the  typical 
indications  and  warnings  scenario); 

•  Estimation  of  second  and  third  order  responses  in  developing  Blue  COAs. 

This  document  summarizes  the  technical  work  done  on  Contract  N00014-14-P-1 185  to  advance 
the  PRELIM  vision.  It  includes 

•  a  short  narrative  that  describes  how  someone  would  benefit  from  using  the  PRELIM 
technology  as  part  of  a  Proactive  Decision  Support  System; 

•  a  technology  roadmap  that  describes  the  science  /  technology  that  needs  to  exist  to  realize 
the  PRELIM  vision; 

•  a  description  of  the  science  research  and/or  theory  that  needs  to  be  done  to  create  make 
these  technologies  real. 


2  Use  Narrative 

One  situation  that  requires  predictive  reasoning  is  making  effective  use  of  digital 
communications  in  DIL  (disrupted,  intermittent,  limited)  bandwidth  environments.  We  present  a 
tactical  use  case  that  shows  the  reality  of  this  requirement  and  illustrates  PRELIM’s  value.  This 
is  only  an  example;  PRELIM  can  be  applied  to  more  strategic-level  decisions  as  well,  and  the 
PRELIM  team  is  eager  to  support  any  of  the  scenarios  discussed  at  the  planning  workshop. 

Many  likely  military  scenarios  require  inserting  troops  into  littoral  regions  that  we  do  not 
control.  The  proliferation  of  Anti-Access/ Area  Denial  (A2/AD)  capabilities  such  as  anti-ship 
cruise  missiles  (ASCMs)  and  anti-ship  ballistic  missiles  (ASBMs)  can  place  our  blue -water 
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assets  at  risk  even  1000  miles  from  shore.  To  respond  to  these  threats,  the  US  Marine  Corps 
(USMC)  has  been  experimenting  with  CONOPS  for  long-range  insertions  via  V-22  and  other  air 
platforms  [3].  The  insertion  flight  can  last  as  long  as  seven  hours,  during  which  events  on  the 
ground  can  make  obsolete  the  situation  awareness  (SA)  and  mission  plans  that  were  current 
when  the  raid  force  boarded  the  aircraft.  The  USMC  has  been  experimenting  with  providing 
troops  on  the  insertion  vehicle  with  tablet  computers  that  can  be  updated  with  SA  during  flight, 
allowing  replanning.  However,  the  comms  channel  to  the  aircraft  (typically  Link- 16)  is 
intrinsically  limited,  and  in  the  case  of  active  hostilities  may  be  further  degraded  by  jamming  or 
interference  with  satellite  communication.  Because  of  this  DIL  environment,  it  is  not  feasible  to 
provide  the  raid  force  with  all  the  information  available  in  the  combat  operations  center  (COC) 
overseeing  the  operation. 

Background:  Consider  the  scenario  in  Figure  1 . 

Two  raid  forces  (RFs)  are  en  route  to  Littoral 
Penetration  Site  (LPS)  1.  RF1  is  to  disable  a 
ASCM  battery  at  OBJ  A,  then  secure  an  airfield  at 
OBJ  B  for  incoming  F35s.  RF2  will  secure  a 
bridge  at  OBJ  C  to  block  Red  reinforcements  and 
maintain  the  transportation  infrastructure  for  later 
Blue  use.  A  previously  emplaced  SEAL  team  has 
deployed  unattended  ground  sensors  (UGS)  in  the 
vicinity  of  OBJ  A. 

During  the  insertion  flight,  these  sensors  report 
strong  vibrations  in  the  vicinity  of  OBJ  A.  This 
MASINT  could  indicate  the  removal  of  the  ASCM  battery,  or  the  arrival  of  strong 
reinforcements,  or  simply  a  civilian  construction  vehicle  moving  over  the  nearby  road. 

Before  PRELIM:  With  no  communications  between  the  insertion  aircraft  and  the  COC,  the  RFs 
land  without  knowledge  of  the  MASINT  and  its  implications,  and  may  face  either  unexpected 
reinforcements  or  a  missing  target  that  has  been  moved  to  another  location.  With  current 
communications  technology  and  without  PRELIM,  the  COC  must  manually  select  which 
information  to  send  to  the  RFs,  and  there  is  limited  ability  to  do  rapid  evaluation  of  second  and 
third  order  Red  responses  in  replanning  the  insertion  mission. 

With  PRELIM:  Table  1  summarizes  how  events  might  unfold  with  PRELIM,  interacting  with 
the  COC  and  the  conventional  Navy  Infonnation  Technology  (IT)  framework.  PRELIM 
monitors  the  mission  plan  against  arriving  ISR,  recommending  that  the  COC  disregard  some 
items,  but  calling  attention  to  the  UGS  MASINT  as  potentially  disruptive.  Its  forecasts  guide  the 
issuing  of  a  priority  infonnation  requirement  (PIR)  to  the  SEAL  team,  then  assesses  the  impact 
of  their  report  to  enable  the  COC  to  replan  and  update  the  Raid  Forces,  so  that  they  land  ready 
for  the  current  situation  on  the  ground. 


Distribution  A:  Approved  for  public  release;  distribution  is  unlimited. 


Page  2 


Final  Report,  Contract  N00014-14-P-1 185 


3  Technology  Roadmap 

In  this  section,  we  review  the  D2REEM  metaheuristic,  which  is  the  core  of  PRELIM,  and 
summarize  its  technical  components.  Then  we  discuss  how  those  components  may  be  realized  in 
the  PDS  program. 
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Table  1:  Evolution  of  insertion  mission  with  PRELIM.  Time  is  relative  to  landing  at  LPS. 


Time 

Event 

Raid  Force 

coc 

Navy  IT 

PRELIM 

-7 

Launch 

Board  V-22 

Makes  go/no-go 
decision 

Continuous 
monitoring  for 
abort  conditions 

Provides  C2  data 
including  current 
plan,  track  feed, 
METOC  forecasts, 
ship  readiness, ... 

Loads  current  mission 
plan;  Monitors  context 
information  (C2); 

Assesses  mission  plan 
against  context 

-6 

Numerous  ISR 
hits  (NTM, 
HUMINT, ...) 

Not  distracted 
with  irrelevant 
traffic 

Reviews  hits 

Link  16 

Determines  that  new 
information  does  not 
impact  current  mission 

-5 

MASINTfrom 

embedded 

SEALS 

identifies 

heavy 

movement  in 
A00 

MASINT  via  Link-16 

RF  1  &  2 
receive  status 
updates 

Notifies  COC 

Recognizes  potential 
mission  impact 

-5 

Review,  approve, 
PIR 

Forecasts  likely  Red 
location  over  time; 

Reports  high-relevance 

IRs  &  affected  futures  to 
COC 

RF  1  receives 
status  updates 

Reports  high-relevance 

IRs  &  affected  futures  to 
Raid  Team 

-4 

PIR  results 
available 

Warning  order 
to  RF  1;  status 
confirmation  to 

RF  2 

Replans  mission 
for  RF  1 

Adjusts  sync 
matrix  for  other 
units 

Link-16  warning 
order  to  RF  1 

-3 

Revised 
mission  plan 
available 

RF  1  revises 
tactics  en  route 

RF  2  reviews 
original  plan 

Reviews  revised 
tactics 

Link-16  between  RF 

1  and  COC 

Monitors  against  revised 
plan 

0 

Insertion 

Both  teams 
land,  ready  for 
the  current 
situation 

Link-16 

Continues  to  monitor  and 
update  force  on  the 
ground 
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3.1  Overall  Roadmap:  the  D2REEM 
Metaheuristic 

PRELIM  is  based  on  the  D2REEM  metaheuristic,  an 
innovative  approach  to  matching  complex  semantic 
models  with  data  [10].  D2REEM  stands  for 
“Dynamic  Data  Relevance  Estimation  by  Exploring 
Models.”  It  supports  any  graphical  model  with  these 
characteristics: 

•  The  edges  express  required  semantic  relations 
among  the  nodes', 

•  Partial  matches  are  meaningful  (the  graph  may 
express  a  range  of  alternatives  of  interest  that  we  wish  to  disambiguate  from  the  data); 

•  A  meaningful  subgraph  corresponds  to  a  trajectory  through  the  graph; 

•  The  probability  that  this  trajectory  visits  a  node  depends  on  evidence  attached  to  the  node. 

Table  2  shows  a  number  of  examples  of  such  graphs, 

Figure  2  outlines  the  overall  flow  of  algorithms  that  implement  this  metaheuristic: 

1 .  The  system  explores  a  graphical  model  to  develop  a  statistical  distribution  over  possible 
trajectories  through  the  model,  reflecting  the  evidence  available  on  the  model’s  nodes. 

2.  It  estimates  the  relative  relevance  of  the  nodes  in  the  model  based  on  this  distribution. 

3.  It  formulates  a  request  for  additional  evidence  for  the  nodes  identified  as  most  relevant. 

4.  It  retrieves  this  evidence,  whether  by  querying  existing  data  or  by  deploying  sensors  to 
gather  additional  intelligence. 

5.  It  updates  the  model  with  the  new  evidence. 

Naive  matching  of  a  complex  model  against  data  is  combinatorially  prohibitive,  but  iteration  of 
the  D2REEM  cycle  retrieves  evidence  in  a  best-first  process  that  is  linear  in  the  size  of  the 
model. 

3.2  Required  Science  and  Technology 

We  outline  the  required  science  and  technology  for  each  of  the  five  components  of  D2REEM, 
and  for  the  antecedent  step  of  constructing  the  model  in  the  first  place. 


Table  2:  Examples  of  Graphical  Models  that  D2REEM  can  exploit 


Graph 

Nodes 

Evidence 

Edges 

Subgraph 

Geospatial  lattice  [6] 

Locations 

Sightings 

Physical  adjacency 

Trajectory 

Narrative  graph  [9]  or 
HMM 

Statements  about  the  world 
“Start”  node 

For  or  against  the 
statement 

Temporal  sequence 

Path  from 
“Start” 

Hierarchical  task 
network  [8] 

Actions 

For  or  against  the 
action 

1 .  Subtasks 

2.  Sequence  constraints 

Path  through 
leaf  nodes 

Social  network  [2] 

People  or  organizations 

Data  on  resource  of 
interest 

Potential  resource  flow 

Path  through 
the  network 

Argument  graph  [12] 

Statements 

For  or  against  the 
statement 

Rhetorical  relations 

Path  through 
the  network 
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Model  Construction. — While  graphical  semantic  models  offer  great  leverage  for  reasoning,  their 
construction  is  often  expensive  and  slow.  A  number  of  technologies  are  available  to  help  reduce 
the  cost  and  speed  the  production  of  model,  including  Bayesian  model  learning  from  examples, 
and  fusion  of  model  fragments  into  a  more  complex  model  [5].  The  “LI”  in  PRELIM  (“Linked”) 
refers  to  the  potential  for  connecting  existing  models  of  different  types  (e.g.,  causal  graphs, 
hierarchical  task  networks,  and  geospatial  representations)  into  more  complex  and  expressive 
structures.  PRELIM  can  readily  process  such  linked  models,  but  automating  the  linking  is  a 
challenging  research  problem. 

Model  Exploration. — A  wide  variety  of  technologies  are  available  to  explore  a  graphical  model 
and  yield  candidate  trajectories.  These  include  probabilistic  inference  (as  in  a  Bayes  network  or  a 
Markov  transition  system),  high-level  semantic  reasoning,  and  Monte  Carlo  sampling.  The 
technology  we  prefer,  polyagent  simulation  [7],  is  a  variety  of  Monte  Carlo  tree  search  [1].  Its 
advantages  include  speed  (the  ability  to  generate  multiple  trajectories  quickly)  and 
expressiveness  (since  each  agent  traversing  the  graph  can  carry  its  own  state  and  behavioral 
preferences). 

Relevance  Estimation. — Given  a  distribution  over  alternative  trajectories  through  the  graph,  we 
seek  to  estimate  the  relevance  of  each  node.  A  node  is  highly  relevant  if  additional  evidence  for 
or  against  that  node  would  make  a  large  change  in  the  distribution  of  trajectories;  it  is  less 
relevant  if  additional  evidence  would  make  little  change  to  the  distribution.  Our  preferred 
approach  to  relevance  estimation  is  genetic  search  over  the  set  of  nodes,  using  information- 
geometric  measures  to  evaluate  alternative  candidate  sets  of  nodes. 

Request  Formulation. — Having  identified  a  highly  relevant  node  in  a  graphical  model,  we  need 
to  formulate  queries  to  available  information  repositories  or  taskable  sensors  that  will  return 
information  to  increase  or  decrease  our  belief  in  the  state  of  affairs  represented  by  the  node,  and 
thus  modulate  the  probability  that  trajectories  will  include  this  node.  The  most  direct  approach  to 
this  task  is  to  include  construction  of  such  queries,  or  of  query  elements  (such  as  relevant 
keywords  or  search  conditions),  in  the  task  of  constructing  the  node  in  the  first  place. 
Alternatively,  candidate  queries  may  be  retrieved  from  the  language  used  in  representing  the 
node. 

Evidence  Retrieval. — Given  a  query,  a  wide  array  of  database  mechanisms  and  sensor  allocation 
schemes  can  be  invoked  to  search  for  the  desired  information. 

Model  Updating. — The  evidence  returned  by  queries  needs  to  be  attached  to  the  nodes  that  led  to 
the  queries  in  the  first  place,  along  with  an  assessment  of  whether  the  query  increases  or 
decreases  the  probability  of  trajectories  that  include  the  node. 

3.3  Realizing  PRELIM's  Technical  Components 

SoarTech’s  efforts  in  PRELIM  focus  on  Model  Exploration  and  Relevance  Estimation  (steps  1 
and  2  of  the  D2REEM  cycle)  to  provide  short-term  forecasts  and  estimates  of  information  value 
in  support  of  Naval  decision-makers. 

PRELIM  works  with  other  performers  in  the  overall  PDS  program  as  a  forecasting  and 
information  valuation  service  that  accepts  world  and  mission  models  with  supporting  data  and 
produces  forecasts  and  relevance  estimates  for  information  not  yet  in  hand  (Figure  3).  The  efforts 
of  other  performers  in  effect  close  the  overall  D2REEM  loop.  In  particular, 
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•  Various  PDS  modeling  services 
will  do  Model  Construction, 
including  Model  Updating 
(D2REEM  step  5)  as  new 
evidence  becomes  available. 

PRELIM  does  include  optional 
tasks  to  explore  the  linking  of 
heterogeneous  model  types. 

•  Various  PDS  performers  are 
focusing  on  presentation 
services.  We  anticipate  that 
human  watchstanders  will  do 
Request  Formulation  (D2REEM 
step  3),  including  information 
source  identification,  for  highly 
relevant  nodes  that  PRELIM 
identifies. 

•  Various  PDS  performers  are 
focusing  on  information 
management,  and  thus  are  in  a  position  to  support  Evidence  Retrieval  (D2REEM  step  4). 


LLI 

CL 

Q. 


Navy  Data  Streams  User  Data  Streams 

Figure  3:  PRELIM  in  the  context  of  the  PDS  program 


4  Opportunities  for  Scientific  Advance 

PRELIM  offers  a  number  of  opportunities  for  significant  scientific  discovery,  summarized  in 
Table  3. 


Table  3:  PRELIM  Objectives  and  Potential  Scientific  Impact 


Information 

Valuation 

1.  Define  &  compute  measures 
of  information  value  that  support 
proactive  collection 
management. 

Develop  an  approach  to  model-based  value  of  information  estimation  that 
is  semantically  richer  than  current  methods 

Develop  visualization  methods  for  information  gaps  that  are  cognitively 
accessible  to  operators 

Prediction 

2.  Generate  distributions  over 
possible  futures  of  the  world  as 
a  function  of  available 
information. 

Demonstrate  ability  to  forecast  over  multiple  model  types  with  a  single 
prediction  engine 

Develop  hybrid  Monte  Carlo/Bayesian  forecasting  methods 

Develop  visualizations  of  forecasts  in  graphical  models  that  allow  users  to 
focus  attention  without  losing  the  big  picture 

Model 

Linking 

3.  Define  formal  linkages 
between  different  model  types. 

Develop  a  canonical  representation  for  semantically  rich  models  that 
subsumes  a  wide  range  of  graph  types 

Develop  mathematical  theory  mapping  model  types  to  one  another 

5  Description  of  All  Tasks  Performed 

Under  this  project,  the  P3  team  has: 


Distribution  A:  Approved  for  public  release;  distribution  is  unlimited. 


Page  7 


Final  Report,  Contract  N00014-14-P-1 185 


•  Prepared  and  presented  workshop  materials,  and  participated  in  a  planning  workshop 

•  Prepared  the  use  narrative,  technology  roadmap,  and  opportunities  for  scientific  advance 
documented  in  the  previous  sections  of  this  report 

•  Filed  the  required  status  reports. 


6  Accomplishments  of  the  Program 

In  collaboration  with  other  members  of  the  PDS  program,  the  P3  project  has  developed  a 
collaborative  vision  (Figure  3)  for  applying  the  D2REEM  metaheuristic  in  the  context  of  a 
broader  architecture  for  proactive  decision  support. 


7  Recommendations  for  Future  Technology  and  Research 

The  P3  team  recommends  that  ONR  develop  a  Proactive  Decision  Support  technology  that 
incorporates  the  technical  insights  presented  earlier  in  this  report,  and  addresses  the  scientific 
questions  detailed  in  Section  4. 


8  Breakdown  of  Contract  Costs 

Labor  and  Indirect  Costs:  $52,578 

Travel:  $3312 

Other  ODCs:  $0 

Fee:  $4052 

Total:  $59,942 


References 

[1]  C.  Browne,  E.  J.  Powley,  D.  Whitehouse,  S.  M.  Lucas,  P.  I.  Cowling,  P.  Rohlfshagen,  S. 
Tavener,  D.  Perez,  S.  Samothrakis,  and  S.  Colton.  A  Survey  of  Monte  Carlo  Tree  Search 
Methods.  IEEE  Transactions  on  Computational  Intelligence  and  AI  in  Games,  4(1):  1-43, 
2012.  http://www.orangehelicopter.com/ed/papers/tciaig  survey.pdf. 

[2]  J.  Crossman,  R.  Bechtel,  H.  V.  D.  Parunak,  and  S.  Brueckner.  Integrating  dynamic  social 
networks  and  spatio-temporal  models  for  risk  assessment,  wargaming  and  planning.  The 
Network  Science  Workshop,  West  Point,  NY,  2009. 
http://abcresearch.org/papers/NetSci2009Crossman.pdf. 

[3]  S.  J.  Freedberg,  Jr.  F-35s,  V-22s,  And  Samsung  Tablets:  Junior  Marines  Pioneer  New 
Tech,  Tactics.  Breaking  Defense,  Breaking  Media,  2014. 

http  ://breakingdefense .  com/20 1 4/05/f-3  5  s-v-22s-and-samsung-tablets-junior-marines- 

pioneer-new-tech-tactics/. 

[4]  G.  Klein.  A  recognition-primed  decision  (RPD)  model  of  rapid  decision  making.  In, 
Decision  Making  in  Action:  Models  and  Methods,  pages  138-147.  Ablex,  Norwood,  NJ, 
1993. 


Distribution  A:  Approved  for  public  release;  distribution  is  unlimited. 


Page  8 


Final  Report,  Contract  N00014-14-P-1 185 


[5]  B.  Li,  S.  Lee-Urban,  G.  Johnston,  and  M.  O.  Riedl.  Story  Generation  with  Crowdsourced 
Plot  Graphs,  the  2  7th  AAAI  Conference  on  Artificial  Intelligence  (AAAI  13),  AAAI, 
Bellevue,  WA,  2013.  http://www.cc.gatech.edu/~riedl/pubs/aaail3.pdf. 

[6]  H.  V.  D.  Parunak.  Real-Time  Agent  Characterization  and  Prediction.  International  Joint 
Conference  on  Autonomous  Agents  and  Multi-Agent  Systems  (AAMAS'07),  Industrial 
Track,  pages  1421-1428,  ACM,  Honolulu,  Hawaii,  2007. 
http://abcresearch.org/papers/AAMAS07Fitting.pdf. 

[7]  H.  V.  D.  Parunak  and  S.  Brueckner.  Concurrent  Modeling  of  Alternative  Worlds  with 
Polyagents.  In,  Multi-Agent-Based  Simulation  VII,  LNCS,  4442,  pages  128-141. 

Springer,  Berlin,  Germany,  2007.  http://abcresearch.org/papers/MABS06Polyagents.pdf. 

[8]  H.  V.  D.  Parunak,  T.  Belding,  R.  Bisson,  S.  Brueckner,  E.  Downs,  R.  Hilscher,  and  K. 
Decker.  Stigmergic  Modeling  of  Hierarchical  Task  Networks.  In  Proceedings  of  the 
Tenth  International  Workshop  on  Multi- Agent-Based  Simulation  (MABS  2009,  at  AAMAS 
2009),  Budapest,  Hungary,  pages  98-109,  Springer,  2009. 
http://abcresearch.org/papers/MABS09rTAEMS.pdf 

[9]  H.  V.  D.  Parunak,  S.  Brueckner,  L.  Downs,  and  L.  Sappelsa.  Swarming  Estimation  of 
Realistic  Mental  Models.  In  Proceedings  of  Thirteenth  Workshop  on  Multi- Agent  Based 
Simulation  (MABS  2012,  at  AAMAS  2012),  Valencia,  Spain,  pages  43-55,  Springer,  2012. 
http://abcresearch.org/papers/MABS2012RMM.pdf 

[10]  H.  V.  D.  Parunak.  Dynamic  Data  Relevance  Estimation  by  Exploring  Models  (D2REEM). 
8th  International  Conference  on  Semantic  Technologies  for  Intelligence,  Defense,  and 
Security  (STIDS  2013),  George  Mason  University,  Fairfax,  VA,  2013. 
http://www.abcresearch.org/papers/STIDS2013D2REEM.pdf 

[11]  N.  Pennington  and  R.  Hastie.  Explanation-based  decision  making:  Effects  of  memory 
structure  on  judgment.  Journal  of  Experimental  Psychology:  Learning,  Memory,  and 
Cognition,  14(3):52 1-533,  1988.  http://psycnet.apa.Org/journals/xlm/14/3/521/. 

[12]  S.  Toulmin.  The  Uses  of  Argument.  Cambridge,  UK,  Cambridge  University  Press,  1958. 


Distribution  A:  Approved  for  public  release;  distribution  is  unlimited. 


Page  9 


